EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria

By simply composing prompts, developers can prototype novel generative applications with Large Language Models (LLMs). To refine prototypes into products, however, developers must iteratively revise prompts by evaluating outputs to diagnose weaknesses. Formative interviews (N=8) revealed that developers invest significant effort in manually evaluating outputs as they assess context-specific and subjective criteria. We present EvalLM, an interactive system for iteratively refining prompts by evaluating multiple outputs on user-defined criteria. By describing criteria in natural language, users can employ the system's LLM-based evaluator to get an overview of where prompts excel or fail, and improve these based on the evaluator's feedback. A comparative study (N=12) showed that EvalLM, when compared to manual evaluation, helped participants compose more diverse criteria, examine twice as many outputs, and reach satisfactory prompts with 59% fewer revisions. Beyond prompts, our work can be extended to augment model evaluation and alignment in specific application contexts

LMCanvas: Object-Oriented Interaction to Personalize Large Language Model-Powered Writing Environments

Large language models (LLMs) can enhance writing by automating or supporting specific tasks in writers' workflows (e.g., paraphrasing, creating analogies). Leveraging this capability, a collection of interfaces have been developed that provide LLM-powered tools for specific writing tasks. However, these interfaces provide limited support for writers to create personal tools for their own unique tasks, and may not comprehensively fulfill a writer's needs -- requiring them to continuously switch between interfaces during writing. In this work, we envision LMCanvas, an interface that enables writers to create their own LLM-powered writing tools and arrange their personal writing environment by interacting with "blocks" in a canvas. In this interface, users can create text blocks to encapsulate writing and LLM prompts, model blocks for model parameter configurations, and connect these to create pipeline blocks that output generations. In this workshop paper, we discuss the design for LMCanvas and our plans to develop this concept.Comment: Accepted to CHI 2023 Workshop on Generative AI and HC

Evolution of the Stethoscope: Advances with the Adoption of Machine Learning and Development of Wearable Devices

The stethoscope has long been used for the examination of patients, but the importance of auscultation has declined due to its several limitations and the development of other diagnostic tools. However, auscultation is still recognized as a primary diagnostic device because it is non-invasive and provides valuable information in real-time. To supplement the limitations of existing stethoscopes, digital stethoscopes with machine learning (ML) algorithms have been developed. Thus, now we can record and share respiratory sounds and artificial intelligence (AI)-assisted auscultation using ML algorithms distinguishes the type of sounds. Recently, the demands for remote care and non-face-to-face treatment diseases requiring isolation such as coronavirus disease 2019 (COVID-19) infection increased. To address these problems, wireless and wearable stethoscopes are being developed with the advances in battery technology and integrated sensors. This review provides the history of the stethoscope and classification of respiratory sounds, describes ML algorithms, and introduces new auscultation methods based on AI-assisted analysis and wireless or wearable stethoscopes

Segmentation and intensity estimation for microarray images with saturated pixels

<p>Abstract</p> <p>Background</p> <p>Microarray image analysis processes scanned digital images of hybridized arrays to produce the input spot-level data for downstream analysis, so it can have a potentially large impact on those and subsequent analysis. Signal saturation is an optical effect that occurs when some pixel values for highly expressed genes or peptides exceed the upper detection threshold of the scanner software (2¹⁶- 1 = 65, 535 for 16-bit images). In practice, spots with a sizable number of saturated pixels are often flagged and discarded. Alternatively, the saturated values are used without adjustments for estimating spot intensities. The resulting expression data tend to be biased downwards and can distort high-level analysis that relies on these data. Hence, it is crucial to effectively correct for signal saturation.</p> <p>Results</p> <p>We developed a flexible mixture model-based segmentation and spot intensity estimation procedure that accounts for saturated pixels by incorporating a censored component in the mixture model. As demonstrated with biological data and simulation, our method extends the dynamic range of expression data beyond the saturation threshold and is effective in correcting saturation-induced bias when the lost information is not tremendous. We further illustrate the impact of image processing on downstream classification, showing that the proposed method can increase diagnostic accuracy using data from a lymphoma cancer diagnosis study.</p> <p>Conclusions</p> <p>The presented method adjusts for signal saturation at the segmentation stage that identifies a pixel as part of the foreground, background or other. The cluster membership of a pixel can be altered versus treating saturated values as truly observed. Thus, the resulting spot intensity estimates may be more accurate than those obtained from existing methods that correct for saturation based on already segmented data. As a model-based segmentation method, our procedure is able to identify inner holes, fuzzy edges and blank spots that are common in microarray images. The approach is independent of microarray platform and applicable to both single- and dual-channel microarrays.</p

The Impact of Depression on Quality of Life in Caregivers of Cancer Patients: A Moderated Mediation Model of Spousal Relationship and Caring Burden

Family caregivers play an important role in managing and supporting cancer patients. Although depression in family caregivers is known to negatively affect caregiver health, the mechanism by which it affects caregivers is not clear. The purpose of this study was to explore the influence of depression on quality of life (QoL) in family caregivers of patients with cancer. Specifically, this study examined (1) whether caring burden mediates the relationship between depression and QoL, and (2) how this mediating effect varies depending on the caregiver’s relationship with the patient. This study performed a secondary analysis on cross-sectional survey data. Ninety-three family caregivers of cancer patients were included in the study. Moderated mediation analyses were conducted using PROCESS macro with the regression bootstrapping method. The moderated mediation models and the indirect effect of caregiver depression on QoL through caring burden were significantly different depending on caregivers’ relationships with patients (i.e., spousal or non-spousal). Specifically, the indirect effect of caregiver depression on QoL was greater for the patient’s spouse than for other family caregivers. Healthcare providers should focus on identifying caregivers’ depression and relationship with the patient and offer tailored support and intervention to mitigate the caring burden and improve the caregivers’ QoL

A patient with a lung adenosquamous carcinoma harboring a de novo T790M mutation and huge nonbacterial vegetative growths successfully treated with osimertinib: A case report

Abstract Nonbacterial thrombotic endocarditis (NBTE) is a rare condition; sterile vegetations attach to heart valves. NBTE is typically found in patients with malignancies or autoimmune disorders. Although surgical interventions are sometimes performed, the appropriate indication and timing are still unclear. Here, we describe a 72‐year‐old woman diagnosed with adenosquamous carcinoma of the lung. She was initially diagnosed as pT2aN0M0 and underwent RUL lobectomy. After nine months, lung cancer recurred, and she underwent treatment with cytotoxic chemotherapy. However, images showed progression after only one month. Rebiopsy revealed she had comutation of de novo EGFR L858R and T790M. Treatment was changed to gefitinib. After one month, she experienced loss of consciousness. Brain magnetic resonance imaging (MRI) showed multiple lesions resembling infarctions or metastases. Chest computed tomography (CT) revealed progression. Osimertinib was prescribed and she underwent echocardiography to rule out the possibility of a cardiogenic embolism. Surprisingly, severe mitral regurgitation and a massive vegetation on the mitral valve were found. Cardiologists recommended surgery due to the severity of the embolic event and valve dysfunction, but it was decided to continue antibiotics, osimertinib, and anticoagulants instead of surgery due to the patient's poor general condition and the possibility of NBTE. Six weeks later, the patient's condition markedly improved and echocardiography revealed a marked reduction in vegetation size. Clinicians should be aware that targeted therapy can be effective in treating severe cancer complications, such as NBTE, as evidenced by the successful treatment of lung cancer with osimertinib. This option should be considered, particularly for elderly lung cancer patients, before resorting to surgery as a first‐line treatment for NBTE

XDesign: Integrating Interface Design into Explainable AI Education

We introduce XDesign, a web-based interactive platform that guides learners through a multi-stage design process for creating user-centered explanations of AI models. Results from a course deployment show that students were able to identify concrete user needs in interacting with explanations, highlight user tasks to support the needs, and design a user interface that aids the tasks. © 2022 Owner/Author

Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph Annotation

Online learners are hugely diverse with varying prior knowledge, but most instructional videos online are created to be one-size-fits-all. Thus, learners may struggle to understand the content by only watching the videos. Providing scaffolding prompts can help learners overcome these struggles through questions and hints that relate different concepts in the videos and elicit meaningful learning. However, serving diverse learners would require a spectrum of scaffolding prompts, which incurs high authoring effort. In this work, we introduce Promptiverse, an approach for generating diverse, multi-turn scaffolding prompts at scale, powered by numerous traversal paths over knowledge graphs. To facilitate the construction of the knowledge graphs, we propose a hybrid human-AI annotation tool, Grannotate. In our study (N=24), participants produced 40 times more on-par quality prompts with higher diversity, through Promptiverse and Grannotate, compared to hand-designed prompts. Promptiverse presents a model for creating diverse and adaptive learning experiences online. © 2022 ACM